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ABSTRACT 


This paper describes the problem-solving studio (PSS) learning environment. PSS was designed 
to teach students how to solve difficult analytical engineering problems without resorting to rote 
memorization of algorithms, while at the same time developing their deep conceptual understanding 
of the course topics. There are several key features of PSS. First, students work in teams of two to solve 
problems, working at the same table with another team of two. The student teams and tables are stable, 
remaining together for most of the semester. The teams work in a public, shared problem-solving space 
that allows in-class mentors (near peers of the students) and the instructor to observe and critique 
their work. The public nature of the work enables the instructor to provide students with real-time, 
situated feedback. In addition, it enables the instructor to tailor the challenge level to the needs of each 
team, such that the problem is too difficult for any one student to solve on their own, but reasonable 
enough that the team can solve it together, given the support that the PSS environment provides. We 
call this targeted adjustment of the problem’s difficulty dynamic scaffolding. PSS provides the sup¬ 
port students need through a specific set of participant structures that govern how the instructors, 
in-class mentors, and students interact during class. This paper describes the PSS approach and how 
it has been implemented in an entry-level course at Georgia Tech called “Conservation Principles of 
Biomedical Engineering (BMED 2210)”. Our results show that even though PSS emphasizes engineering 
problem-solving skills, the students' conceptual understanding of the material significantly improves, 
as measured by Shallcross’ material and energy balances concept inventory. The implications of our 
work, particularly with respect to the use of PSS in flipped classrooms, are discussed. 

Key words: studio, problem-solving, problem-solving studio, flipped classroom, dynamic scaffolding, 
collaborative learning, apprenticeship, engineering, material and energy balances, near peers 
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INTRODUCTION 


A key objective of many first-year engineering courses is to introduce undergraduates to the 
approaches engineers take to solve problems. In an effort to apprentice students to the engineering 
problem solving approach, many entry-ievel engineering textbooks provide students with a step-by-step 
procedure that is remarkably similar across multiple domains of engineering (Craig 1996; Himmelblau 
and Riggs 2004; McGill and King 1995). These procedures encourage the problem solver to 1) read and 
understand the problem, 2) abstract the key features of the problem to create an idealized model of 
the system, often in the form of a diagram, 3) use the diagram, in concert with fundamental principles, 
to generate and solve a mathematical model, and 4) check whether or not the answer is reasonable. 
These steps are explicitly listed and explained in the textbooks and course instructors emphasize and 
teach these steps to their students. Nevertheless, large numbers of students continue to struggle in, 
and sometimes fail to pass, entry level engineering courses (Felder et al. 1994; Suresh 2006). 

One reason for this is that many novice engineering students are ill-equipped to solve engineering 
problems. Many students enter their first engineering course adept at solving relatively simple, well- 
structured problems but with little prior experience in dealing with ill-structured, complex problems 
(Hestenes, Wells, and Swackhamer 1992; Rosengrant, Van Heuvelen, and Etkina 2009; Jonassen and 
Hung 2008). As a result, most students come to their first engineering course with a well-practiced 
strategy that uses a rote problem solving approach in which they 1) write down the known and un¬ 
known variables they find in the problem statement, 2) search for a formula or equation that uses 
these variables, and then 3) enter the numbers into the formula and calculate an answer (Rosengrant, 
Van Heuvelen, and Etkina 2009). Indeed, a common lament of engineering professors is that their 
students persist in using ineffective “plug-and-chug” approaches in which they search for problems 
with similar surface features to the one they are trying to solve and then plug their numbers into the 
equations that worked for these similar problems, hoping this will lead to the correct answer. Little or 
no attention is paid to step two of the textbook problem-solving method which recommends creat¬ 
ing an idealized diagrammatic model of the system described in the problem statement and using 
the diagram to generate the relevant mathematical equations (Bella 2003; Waller and Le Doux 2014; 
Waller, Newstetter, and Le Doux 2013; Waller, Le Doux, and Newstetter 2013). 

Similar observations have been made in chemistry courses. Several studies have shown that the 
majority of students in high school and college chemistry courses rely almost exclusively on an algo¬ 
rithmic approach to quantitative problem solving and that experience with an algorithmic approach 
does not facilitate conceptual understanding (Anamuah Mensah 1986; Bunce, Gabel, and Samuel 1991; 
Gabel, Sherwood, and Enochs 1984; Herron and Greenbowe 1986; Lythcott 1990; Niaz and Robinson 
1992). Cracolice et al., in a study of students’ algorithmic and conceptual problem-solving abilities in 
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a first-semester general chemistry course, concluded that students have no choice other than to be 
algorithmic problem solvers because their reasoning skills are not sufficiently developed to allow them 
to successfully solve conceptual problems (Cracolice, Deming, and Ehlert 2008). The authors noted that 
despite an increased emphasis in the past several years on conceptual understanding and connecting 
concepts to problem-solving in textbooks and curricula, the algorithmic-conceptual gap has remained 
relatively constant. Bodner, as far back as 1992, suggested that solving the algorithmic-conceptual 
gap can not be achieved through curriculum changes alone; rather, changes must be made in how the 
curriculum is delivered (Bodner 1992). That is, if we want students to learn how to become more effec¬ 
tive problem solvers, who do not rely on rote memorization of procedures, formulas, and algorithms to 
solve problems, than we need to design and implement more effective learning environments. 


OVERVIEW OF THE PROBLEM-SOLVING STUDIO 


Development of the problem-solving studio 

To address this need, a powerful learning environment we call the problem-solving studio (PSS) 
was developed by one of us (J.L.) for teaching engineering ways of thinking and analytical problem 
solving skills. The PSS approach refers to a specific set of participant structures that govern how 
the instructors, in-class mentors, and students interact during class, with each other and with the 
objects of learning (Philips 1972). PSS is silent with respect to what activities, if any, the instructor 
should require their students to engage in outside of class. PSS was first implemented in 2008 in 
a large biomedical engineering department as an experimental approach for teaching a required 
entry-level course called “Conservation Principles of Biomedical Engineering” (aka BMED 2210), then 
further developed over several years via an informal design research process in which changes to the 
learning environment were made based on the professor’s observations, feedback from his students, 
and in consultation with a learning scientist who was a full-time faculty member in his department. 
More recently, we have begun to carry out formal research studies of the learning environment to 
deepen our understanding of its key features and how they affect students’ motivation and learning. 

As the PSS approach was being developed, advances in technology and multimedia infrastruc¬ 
ture stimulated significant interest in the last two to three years in using the “flipped classroom” 
to teach engineering courses (Jahnke and Norberg 2013; Ng 2015a). The term flipped classroom, 
in its broadest sense, refers to a learning environment in which the professor puts their lectures 
online, thereby freeing up classroom time for other learning activities which are intended to be more 
interactive and engaging (Ng 2015b; Larcara 2014; Gardner 2015). The term flipped classroom says 
nothing, however, about the nature of these in-class activities or how they are structured, other than 
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to suggest that they are not full lectures. The PSS learning environment potentially fills this void 
because although PSS does not require instructors to put their lectures online, it is a useful model 
of how they can structure the in-class time they have freed up by putting their lectures online. So, 
in summary, the term flipped classroom specifically refers to the out of class activity of requiring 
students to view online lectures, whereas we use the term problem-solving studio (PSS) to refer to 
how we structure the time we have with our students in the classroom. 

The problem-solving studio and the architecture design studio: a comparison 

The name “Problem-Solving Studio” was chosen because the PSS approach shares several sig¬ 
nificant features with the architectural design studio. It is useful, therefore, to briefly reflect on the 
similarities and differences between the two approaches. The architecture design studio tradition 
was first established at the Ecole des Beaus-Arts in Paris, an influential educational institution that 
was founded in 1648 (Ockman and Williamson 2012). Currently, architecture students often begin 
their studies of design in their first year, in studios that are led by a studiomaster, who is assisted by 
studio teachers, or critics, who oversee the students’ work in a particular section of the large room 
(i.e., studio) where all the students of the same level work side-by-side (Dinham 1987). Students’ 
work centers on design problems, which takes precedence over lectures as the primary teaching 
tool (Ockman and Williamson 2012). Student work on a project begins when they are handed a 
one to five page project description, after which they spend their time studying the requirements, 
planning general approaches, and making sketches of their solutions that they iteratively develop 
(Dinham 1987). Their work culminates in their solution to the design problem, which often includes 
a number of drawings and a physical model. 

The primary means by which architecture students receive feedback during the course of a 
project is through the desk crit. The desk crit is a brief event that occurs repeatedly on any given 
day in the studio. The critic moves through the studio room and meets with students, one-on-one, 
either randomly or through informal appointments, to discuss their work (Dinham 1987). The critic 
also meets with groups of students to discuss points that are relevant to all students. Occasionally, 
perhaps one to three times during the course of a project, all the students will pin up their work on 
a wall and the critic will move from one student’s work to the next, critiquing their work publicly in 
front of all the students (Dinham 1987). Students present their final projects to a jury of faculty and 
local practitioners who also critique their work in front of the other students, who are expected to 
learn from these reviews (Ockman and Williamson 2012; Dinham 1987). 

PSS shares several significant features with the architecture studio. As with architecture students, 
engineering students first experience the PSS environment early in their undergraduate careers. Their 
primary activity in PSS is solving engineering problems, work which takes precedence over the lecture 
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as the main method for teaching. The importance of creating sketches and engineering diagrams and 
using them as thinking tools is heavily emphasized. In addition, the students’ work is carried out in a 
public space (that is, it is visible to other students and the instructor) and the instructor frequently 
enters into a discussion with them about their work. These discussions are what Ruiz-Primo calls 
“informal formative assessments” in which the instructor seeks to make a student’s “thinking explicit 
in an unobtrusive manner so that it can be examined, questioned, and shaped as an active object of 
constructive learning” (Ruiz-Primo 2011). The instructor does this by asking open-ended questions to 
get the students talking about their problem solving processes, such as “How is it going?” or “What 
assumptions did you make and why?” or “Can you summarize for me what you have done so far?” In 
addition, older students, who performed well in the course in a prior semester, serve as in-class men¬ 
tors and roam the room to assist the instructor in providing feedback to the students. Similarly, at the 
Ecole des Beaus-Arts, it was common practice for the older students (the anciens) to help the newer 
students (the nouveaux) (Ockman and Williamson 2012). The in-class mentors are not formally trained 
for their role, but the instructor mentors them during PSS and during as-needed meetings outside of 
class. Finally, both use the apprenticeship model of learning, in which the novice learns by observing 
the expert undertake a specific task, and then attempts the same task while getting feedback and 
guidance from the expert (Collins, Brown, and Newman, 1989). In PSS, the students are engaged in a 
cognitive apprenticeship with their instructor because the tasks that are being learned are intellectual, 
not physical, in nature (Newstetter 2005). Key to this kind of instructional relationship is the support 
the expert provides in the form of observations, evaluations, and just-in-time help (Newstetter 2005). 
The expert gradually reduces the amount of support they provide over time as the novice begins to 
master the task, ultimately leading to the novice working alone (Newstetter 2005). 

Although PSS and the architecture design studio approaches have a number of similarities, there 
are some significant differences. First, in PSS, students work on solving analytical engineering prob¬ 
lems, not design problems. Second, the students work in teams of two at a table with another team 
of two students to collaboratively solve the problem. Third, students’ work is process-oriented, not 
product-oriented. The fundamental goal of the teamwork is to learn good problem solving pro¬ 
cesses, not to produce a specific design, product, or report. When students’ knowledge is assessed 
via quizzes and tests, their work is graded based on the process they used to solve the problems, 
with little weight given to whether or not they ultimately came up with the correct final answer. 

The problem-solving studio and other active learning environments: a comparison 

Later in this paper, we elaborate on the details of the problem-solving studio but it is useful to 
compare its general characteristics to other active and cooperative learning environments (see 
Table 1). Several features distinguish it from formal cooperative learning, Process Oriented Guided 
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PSS 

Informal 

Cooperative 

Learning 

Formal 

Cooperative 

Learning 

POGIL 

Scale-up 

Laboratory 

Courses 

Environment 

classroom with 
tables; pairs of 
students on each 
long side 

lecture style 
room 

various 

classrooms; 

perhaps 

tables 

various 
classrooms; 
perhaps tables 

classroom 
with tables for 

9 people and 

3 computers 

laboratory 

Length of 
engagement 

1.5-2 hours per 
class; 25 classes 
per semester 

2-10 minutes in 
class between 
mini-lectures 

several class 
days to full 
semester 

50 minute class 
period 

50-120 
minutes per 
class 

2-3 hours per 
lab; 5-12 labs 
per semester 

Activities 

students solve 
complex 
problems that 
engage multiple 
concepts 

students develop 
short answers 
to questions, 
brainstorm, 
solve small 
problems 

students work 
together on 
large projects 

students work 
to complete 
worksheets 
that guide them 
through the 
process of learning 

hands-on 
activities, 
simulations, 
or interesting 
questions and 
problems 

students follow 
instructions 
to reproduce 
experiments 
with known 
results 

Size of 
groups 

2 students per 
team; 2 teams 
per table 

usually pairs 

small groups 
of 2-6 
students 

2-6 students 

3 teams of 3 
at each table 

2-3 students 

Role 

assignment 

students 

negotiate 

together 

if roles are 
used, randomly 
assigned 

students are 
assigned roles 
that rotate 

students are 
assigned roles that 
rotate 

not specified 

student 

negotiate 

together 

Problem¬ 
solving space 

blotter pads 
shared by 2 
students 

individual 

notebooks 

individual 
or shared 
notebooks 

each student has 
copy of worksheet; 
one team solution 
submitted 

each team 
shares a 
computer 

lab bench with 
equipment; 
individual lab 
notebooks 


Table 7. A comparison of PSS to other similar learning environments. 


Inquiry Learning (POGIL), Scale-up, and lab courses (Crouch & Mazur, 2001; Heller, Keith, & Anderson, 
1997; D. W. Johnson, Johnson, & Smith, 1998b; 1998a; Moog, Creegan, Hanson, Spencer, & Strauma- 
nis, 2006; Richards, Kustusch, Ding, & Beichner, 2008; Smith, 2000). The students are partnered 
all semester rather than for a single class or project. Although they are given complex problems to 
solve, they are not doing guided inquiry via worksheets or lab procedures. Students in PSS share 
the writing task, but roles are negotiated between the students rather than being assigned. In both 
PSS and Scale-up, the instructor circulates to assist the students, but PSS also includes interactions 
with near-peer mentors. In Scale-up, student teams share a computer, while in PSS students work 
together on a shared large pad of paper. This creates a public, shared problem-solving space that, to 
our knowledge, is unique to PSS. In summary, although PSS incorporates some features of different 
instantiations of cooperative learning, its integrated components set it apart. 

Description of the entry-level biomedical engineering course used to develop the PSS approach. 

Next, we describe the course we used to develop the PSS learning environment, including its 
list of topics. BMED 2210, “Conservation Principles of Biomedical Engineering”, is usually among 
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the first required engineering courses that a biomedical engineering student takes at Georgia Tech. 
It is a 4-credit hour course. To register for the course, students must have earned a C or better in 
both Physics I and General Chemistry I. Physics I exposes students to Newton’s laws and the prin¬ 
ciples of momentum and energy conservation. General Chemistry I covers the fundamental laws 
and theories of chemical reactions. In combination, these pre-requisite courses provide students 
with a solid grounding in the fundamental principles of physics and chemistry needed to succeed 
in the course. Most students take this course in their 2 nd year in college. Although the focus of this 
paper is BMED 2210, we believe that PSS is appropriate for use in any level problem-solving course. 

The purpose of BMED 2210, as stated in the syllabus, is to prepare students to “analyze and solve 
problems involving complex biological systems by applying principles of mass and energy conserva¬ 
tion. More fundamentally, this course introduces students to the engineering approach to problem 
solving. This includes breaking a system down into its components, establishing the relationships 
between known and unknown system variables, assembling the information needed to solve for the 
unknowns, then obtaining the solution.” By the end of the course, students are expected to know the 
basics of conducting engineering calculations and be able to apply these concepts and principles 
to the analysis of physiological systems. The primary textbook for the course is “Bioengineering 
Fundamentals” by Saterbak, Mclntire, and San (SMS) (Saterbak, San, and Mclntire 2007). Students 
are also encouraged to consult Himmelblau and Rigg’s (H&R) “Basic Principles and Calculations 
in Chemical Engineering, 8th edition” as a supplemental text (Himmelblau and Riggs 2004). The 
topical outline of the course is provided below (Table 2). 


Week Topic 

1 Introductory concepts: units, significant figures, properties, estimation, diagrams 

2 Material balances for single-unit, non-reacting systems 

3 Material balances for multi-unit, non-reacting systems 

4 Material balances for single-unit, reacting systems 

5 Material balances for multi-unit, reacting systems 

6 Material balances for systems with gases: ideal gas law 

7 Material balances for systems with multiple phases: phase diagrams and vapor pressure 

8 Material balances for systems with multiple phases: saturation and partial saturation 

9 Introduction to energy balances: terminology, types of energy that must be accounted for 

10 Calculating changes in energy at the molecular level: internal energy and enthalpy 

11 Energy balances for non-reacting systems 
12/13 Energy balances for reacting systems 

14 Material and energy balances for non-steady-state systems 

15 Review 

Table 2. Topical outline of BMED 2210. 
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DETAILED STRUCTURE OF THE PROBLEM-SOLVING STUDIO 


To support the learning of these concepts and skills, we immerse our students in the PSS learning 
environment, which is distinctive in a number of significant ways. First, although the contact time 
is the same for PSS as for the lecture-based version of the course (4 hours), the PSS class meets 
less often for longer periods of time: 2 times for 2 hours each in PSS versus 4 times for 1 hour each 
in the lecture-based course. 

Physical Layout of the Course 

In addition, unlike a traditional course where the professor uses most of their contact time with 
the students to lecture, PSS employs a number of different kinds of interactions. To understand how 
these interactions take place, it is important to understand the physical setting of the room, and 
how the participants and learning materials are organized. The first major difference one observes 
is the physical setup of the room. In the Biomedical Engineering Department at Georgia Tech we 
have outfitted two classrooms to support PSS. Each classroom can hold up to 48 students seated 
at 12 tables. All the tables and chairs are on wheels. Ideally, the furniture is configured so that each 
table is isolated so that there is sufficient space between the tables to make it easy for the instructor 
to move about the classroom and to quickly reach any student or table. There are several white¬ 
boards distributed among the walls of the room that are magnetized so that the students’ work 
can be posted for review and discussion. This setup allows the instructor and students to configure 
the room in ways they believe best supports their learning for that particular day's activities. It is 
important to note that this specific setup is not required to implement PSS. PSS can be used in 
any kind of classroom as long as students can work in teams of two and the instructor and in-class 
mentors are able to observe the team’s work and carry out a desk crit. Nevertheless, it is prefer¬ 
able to implement PSS in a classroom that can be reconfigured as needed to optimally support the 
planned learning activities. 

Students solve problems that are more complex and less structured than typical textbook problems 

Having described the physical configuration of a typical PSS classroom, we will now describe 
what the classroom looks like when students are working together to solve an engineering problem. 
The primary purpose of BMED 2210 is to develop the students’ analytical problem solving skills. It 
is not a design course. Nevertheless, for the most part, we give students problems that are chal¬ 
lenging enough that the students solve only one to three problems in a typical 2-hour PSS class 
session. We find it helpful to use Jonassen and Hung’s problem difficulty classification scheme to 
describe the kinds of problems we ask PSS students to solve (Jonassen and Hung 2008). Jonassen 
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and Hung propose that a problem’s difficulty is a function of its complexity and its structuredness. 
The complexity of a problem manifests itself in “the breadth of knowledge required, the difficulty 
level of comprehending and applying the concepts involved, the skill and knowledge levels required 
to solve the problem, and the degree of nonlinearity of the relations among the variables within 
the problem space”. The factors that characterize the structuredness of a problem include “known 
versus unknown states of the problem, regular versus unconventional uses of rules and principles 
involved, stated constraints versus hidden constraints, predictable operators versus unprescribed 
operators, a preferred and prescribed solution versus multiple viable solutions, and definite versus 
vague criteria for evaluating the solutions” (Jonassen 1997). 

Using Jonassen and Hung’s terminology, the problems we ask our students to solve in PSS are 
reasonably well-structured but somewhat complex. We frequently base the problems we give our 
students on problems from their textbooks, but we often modify them to make them more ill- 
structured and complex. Generally, we make the problems we assign more ill-structured by reduc¬ 
ing their transparency by increasing the number of unknowns and by assigning or altering problem 
statements to increase the number of possible ways to solve the problem (that is, we increase the 
number of “legitimate competing alternatives”) (Jonassen and Hung 2008). In addition, we steadily 
increase the complexity of the problems throughout the semester, particularly with respect to the 
intricacy of the solution process. That is, as the semester progresses, we tend to use problems with 
longer path lengths from the initial state to the goal state of the problem and we increase the number 
of relations and concepts the students need to process while solving the problems. The reduced 
structure and increased complexity of these problems provides students with multiple options 
for how to approach and solve them, which leads to robust discussions among the students and 
instructors. Please see Table 4 in the appendix for a detailed illustration of this approach. 

Dynamic scaffolding is used to provide students with the appropriate level of challenge 

According to Jonassen, because the PSS problem is less well-structured and more complex than 
the textbook problem, it will be more difficult to solve (Jonassen and Hung 2008). The instructional 
challenge is to present each team, each table, and the entire class with a problem that is appro¬ 
priately challenging, but not so difficult that students make little to no progress for an extended 
period of time. Vygotsky conceptualized the difference between what an individual can do on their 
own versus what they can do with the help of a more knowledgeable other, such as a peer, instruc¬ 
tor, or through some other kind of instructional support (aka scaffolding), as the zone of proximal 
development (ZPD) (Vygotksy 1978). The role of the instructor, then, is to provide support and as¬ 
sistance to enable the student to operate at a higher level than they could if they were working on 
their own. The support that the instructor or the learning environment provides is called scaffolding 
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Problem 

difficulty 



Student and team's skill level 

Figure 1. The relationship between problem difficulty, student’s skill level, and the zone of 
proximal development (ZPD). 


(Wood, Bruner, and Ross 1976). Ideally, students are operating within their ZPD; otherwise, they will 
become confused and discouraged if the work they are doing is too far beyond their capabilities, or 
bored and disinterested if the work they are doing is too beneath their current capabilities (Figure 1). 

Typically, students within a class are at many different levels of development for any given skill, 
concept, or problem. In an ideal learning environment, each student would be operating within 
his or her own ZPD. This is difficult to achieve in a traditional lecture-based course because the 
instructor can deliver only one lecture at a time, which may be at a level that is appropriate for 
only some of the students (Exley and Dennick 2004). If there are differences in students’ compre¬ 
hension, the lecturer is not likely to detect or respond to this since the students’ thinking is not 
made visible in a traditional lecture setting. In contrast, several features of PSS make it possible 
to dynamically modify in real-time the scaffolding that is provided to the students, either at the 
local level of an individual student-team or table, or at the global level of the entire class, in order 
to keep as many students as possible within their zone of proximal development (ZPD). We call 
this dynamic scaffolding. 

An instructor can dynamically scaffold students’ learning by ratcheting up or down the difficulty 
of a problem, by making it less or more complex or by making it less or more structured, as described 
above. Typically, in PSS, the instructor would begin by presenting the entire class with a problem 
that they believe is sufficiently difficult to be within the ZPD of the class’ more advanced students. 
The instructor would then need to assess, in real-time, the progress that students are making on the 
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problem to determine if, and for whom, the problem’s difficulty needs to be modified. The instruc¬ 
tor could choose to reduce or increase the difficulty of a problem for a single team or table, or for 
the entire class. 

The problem-solving studio uses multiple participant structures to support student learning 

PSS enables instructors to monitor students’ progress in real-time through the participant 
structures it creates. Susan Philips defined “participant structures” as the “ways of arranging verbal 
interaction with students, for communicating different types of educational material, and for pro¬ 
viding variation in the presentation of the same material” (Philips 1972). Philips found that student 
learning can be profoundly impacted by the participant structures that are used, both because they 
strongly influence how students interact and communicate with each other and with the instructor, 
and because some learning objectives are best achieved through "one sort of participant structuring 
rather than another” (Philips 1972). 

The primary participant structure of PSS is the team of two students who problem-solve together 
on a publicly visible problem-solving space (we typically use 17”x22” pads of blotter paper). The 
instructor challenges the students with a problem, either by describing the problem statement orally, 
writing it on a whiteboard, or by handing them a printed problem statement. The student pairs then 
begin working on the problem together. In most cases, one student writes on the blotter pad while 
explaining what they are doing to their partner. The partner who is not writing actively engages in 
the problem-solving process by listening carefully, agreeing with or critiquing what their partner is 
doing, and suggesting their own ideas about how to proceed. Every few minutes the students switch 
who is holding the pen. The students negotiate who holds the pen and for how long. 

There are three key features of this participant structure that we believe promote learning (see 
Figure 2). First and foremost, it requires students to explain and defend the approaches they take to 
solve the problem. Self-explanation such as this promotes learning and facilitates problem solving 
by helping the problem solver draw conclusions and make inferences from the problem statement 
when critical information is missing (Wickelgren 1974; Chi et al. 1989). Second, the two students must 
work together to solve the problem. This requires students to argue their points, to communicate 
clearly and persuasively, and to negotiate with a peer which route to take when solving the prob¬ 
lem (John-Steiner and Mahn 1996; Doolittle 1995). Finally, the third key feature of this participant 
structure is that the team’s work is publicly visible, to the other team at their table, as well as to the 
in-class mentors and the instructor. 

The second participant structure of PSS is when the pair of teams that are seated together at 
a table confer with each other to solve the problem. The teams form a kind of distributed cogni¬ 
tive network in which they leverage each other’s skill sets and knowledge bases to improve their 
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Figure 2. The key participant structures of PSS. Panel A shows students working on blotter 
pads in teams of 2, across from another team of 2 at the same table. Panel B shows an in-class 
mentor, a “near-peer”, working with a student team. Panel C shows the instructor mentoring a 
table of 4 students. 


ability to interpret the problem and navigate their way towards a solution (Hutchins 2006). The 
student teams confer with each other when they find themselves struggling with a problem, which 
effectively expands their capabilities, stretching their collective zone of proximal development a bit 
further. Importantly, the students work together in the same teams and at the same table for most 
of the course. Students choose their own teams initially, but two to three weeks into the course 
the instructor assigns permanent teams. The teams are heterogeneous in terms of the students’ 
scores on the weekly quizzes and the early concept inventory (which we discuss in more detail 
later), although each student is given the opportunity to anonymously identify one student they 
want to be at their table. We have found that giving students a say in who will be at their table 
reduce their resistance to this change, and in fact many students welcome the change. Stable 
teams of this size, sometimes called cooperative base groups, enables students to develop a sense 
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of trust in, and responsibility to, each other, which improves the quality and quantity of learning 
(D. W. Johnson & Johnson, 1999). 

The third participant structure of PSS is when an in-class mentor (a near peer) or the instructor 
interacts with a team or table of students, initiated by the students or by the mentor/instructor 
(Liberatore, 2013; Prather, Rudolph, & Brissenden, 2011). The students raise their hands when they 
want to interact with an in-class mentor or the instructor. Most often, students request this interac¬ 
tion when they have a question about the problem statement, want to see if they are on the right 
track, are not sure how to proceed, or want feedback on their solution. Sometimes the instructor or 
in-class mentors initiate this structure without waiting for the students to request it. The instructor 
forms this structure to thoroughly assess how the students are doing, to determine if their approach 
is sound or not, and to see if they have any significant misconceptions. While in this structure, the 
in-class mentors and instructors do not directly teli the students what to do or how to proceed, nor 
do they perform any calculations or problem solving for the students. Rather, they primarily interact 
with the students by asking questions, to model for them the kinds of probing questions experts ask 
themselves as they progress through a problem. Even if the students have completed the problem 
and have the correct numerical answer, the in-class mentors and instructor thoroughly review, with 
the students, the approach the students took to solve the problem to ensure they understood the 
principles that were in play and to ensure that the problem-solving approaches they took were 
sound, before they release the students to begin working on the next problem in the problem set. 

The discourse that takes place within this participant structure may differ, depending on whether 
the instructor is an in-class mentor or the professor (Chi & Wylie, 2014). This is because the professor 
has more experience and expertise in the topics of the course than the in-class mentors. This can be 
advantageous, because the instructor may be better prepared to troubleshoot student solutions, 
and better able to guide students in discussions about the deep principles that underlie the problem. 
The professor’s expertise can also be a disadvantage under some circumstances, because much 
of their knowledge will be tacit, which is knowledge that is unconscious and can not be written or 
verbalized. In contrast, less of the in-class mentors’ knowledge, having recently taken the course, 
will be tacit (Polyani 1966; M. Chi, Glaser, and Rees 1982). They are skilled participants, not experts. 
These skilled participants serve as “near peer” role models, which in some circumstances may be 
better suited than the expert to coach the students with questions because they may find it easier 
to verbalize their ways of thinking to the students. According to Murphey, near peers are “more 
psychologically attractive” to students because their “excellence seems more possible and easy 
to see and replicate because they are in some ways already very similar to” them (Murphey 1996). 

This process, in which the instructor and student reflect on the challenges the student encoun¬ 
ters while solving a problem, and take action based on that reflection to enhance learning during 
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the learning process, is called formative assessment (Cowie & Bell, 2010). PSS makes possible the 
dynamic formative assessment of the student teams by the fact that the teams are working together 
in a shared problem-solving space on the desk blotter pads. This ability to dynamically assess 
student work in real-time is critical to the success of PSS, because as it is taking place, both the 
students and the instructor are receiving situated feedback. The students are getting feedback on 
their problem-solving approach and the instructor is getting feedback on how well the students are 
performing and how they are experiencing the problem. In parallel, the instructor, after observing a 
small number of teams, can gauge whether or not specific teams, or perhaps the entire class, need 
additional support to progress through the problem. This situated feedback enables the instructor 
to decide, in real-time, whether or not to ratchet up or down the problem difficulty, for one team 
or table, or for the entire class. In addition, PSS instructors learn very rapidly what the conceptual 
difficulties are that students struggle with most, and what kinds of problems and lines of question¬ 
ing best help them overcome these challenges. We believe that PSS instructors, because they see 
so much of their students work, will make rapid advances in their pedagogical content knowledge. 
Pedagogical content knowledge is a key component of teaching expertise and, according to Shulman 
“includes an understanding of what makes the learning of specific concepts easy or difficult: the 
conceptions and preconceptions that students of different ages and backgrounds bring with them 
to the learning” (Shulman 1986). 

The fourth participant structure is when the instructor interacts with all the students at the same 
time. Typically this is used only 3 or 4 times during a 2-hour class period for 3 to 5 minutes at a time. 
It is usually implemented when the instructor sees that multiple groups are struggling with similar 
issues. The instructor most often uses this structure to carry out a "just-in-time discussion”, which 
the students are highly motivated to participate in because they are engaged with, and challenged 
by, the problem and are hungry for insights that will help them figure out how to get past the intel¬ 
lectual obstacles that are blocking their progress (Schwartz and Bransford 1998). Often, the instructor 
uses these discussions to draw students’ attention to the key concepts or skills needed to solve the 
current problem or to model problem solving strategies and heuristics. In addition, the instructor can 
use this time to develop students’ metacognitive skills by teaching them the importance of reflect¬ 
ing on their performance, helping them identify which concepts and problem-solving approaches 
that they still need to work on, and encouraging them to create an action plan to overcome these 
intellectual shortcomings. 

We are often asked how we encourage students to engage in the problem-solving studio envi¬ 
ronment and if some students resist. Our experience has been that the vast majority of students 
participate right away, enthusiastically. There is an initial “training period” during the first week or 
two, during which we remind the students at the beginning of class to use their blotter pads, to work 
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in teams of two at tables of four, and to share the pen. But after the first few meetings, the students 
begin doing this automatically, without being reminded. We believe that the students are engaged 
because they feel safe and supported: they are not being graded, they work with a partner at a 
table with three other students, and they get real-time assistance from the in-class mentors and the 
instructor whenever they ask for it. In addition, their work is public, which motivates the students to 
remain engaged since the in-class mentors and the instructor can, and often do, join their group to 
carry out an impromptu desk crit. We find that for a class of 48 students, two in-class mentors and 
one instructor are needed to provide sufficient real-time support for the students. 

Multiple types of formative assessment are used to assess student learning 

An important feature of PSS is that it clearly demarcates formative assessments from summative 
assessments (i.e., graded activities) of student learning. In PSS, except when the instructor is ad¬ 
dressing the entire class, formative assessment is happening nearly continuously for every student in 
the class. This is made possible because the students are working together, explaining their thinking 
processes to each other out loud and co-constructing solutions to the problem they are working on 
in a shared problem-solving space that is public. The instructor and in-class mentors roam about 
the room and listen to and see each team’s approach by strolling near their table and eavesdrop¬ 
ping or by looking at what they have written on their blotter pad. Instructors and in-class mentors 
often then initiate a dialogue with the team or table they are observing to challenge the team on 
the approach they are taking, to learn more about how they are thinking about the problem, or to 
ask them what difficulties or challenges they are encountering. These conversations are primarily 
propelled through questions that the instructor asks the students, questions that are open-ended 
but targeted with the intent to stimulate the students’ metacognitive thinking and to probe specific 
aspects of their knowledge relevant to the problem they are solving. These conversations frequently 
reveal student misconceptions, even when a student team appears to be on the right track, and pro¬ 
vide the instructor with the information needed to help the students recognize their misconceptions 
and to adjust the problem difficulty as needed (i.e. to dynamically scaffold). It is important that the 
students know they are not being graded or judged at this point. This establishes the studio time 
as a safe time for students to explore, to challenge themselves, to admit they don’t understand, and 
to seek help by asking questions. 

In addition to the real-time feedback students receive while solving problems with their team¬ 
mates in PSS, we also give students the opportunity to experience a quiz-like setting in which they 
are given a defined period of time to solve a problem on their own first, before discussing it with 
their teammates. These practice quizzes are given immediately after students turn in that week’s 
homework, and they are intended to provide students with feedback on how well they understand 
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the homework material. Briefly, on practice quiz days, students turn in their homework and then 
immediately begin working on a practice quiz. Most of the time, the practice quiz is an old quiz that 
was given to a previous semester’s class as a graded assignment. The students work on the practice 
quiz on their own, as if it were a real quiz, for about 20 minutes, after which they discuss the quiz 
with their tablemates for about 5 minutes. Finally, a class discussion is held about the practice quiz 
between the students and the instructor, until the students’ questions about the quiz are answered. 
The practice quizzes give the students feedback about their readiness for the graded quiz, which is 
taken two days later during the next class period, and it models for the students what they should 
be doing on their own, which is to periodically test themselves, under a quiz or test-like setting, to 
evaluate how well they know the material before they are formally evaluated in class on a graded quiz 
or test. Several recent studies suggest that pre-testing students in this way may lead to significant 
gains in learning (Agarwal et al. 2008; Bjork and Storm 2011). 

Out-of-class learning assignments and summative assessments of student learning 

PSS is an approach for how to structure in-class learning activities that provides students with 
copious amounts of formative, real-time feedback, usually in a group setting. PSS does not require 
particular out-of-class activities nor does it determine how student learning should be assessed. As 
a result, there is a wide range of possible ways PSS instructors can further support student learning 
through out-of-class learning activities and through summative assessments for grades. However, 
these out-of-class activities and summative assessments should be congruent with the PSS ap¬ 
proach. For example, grading exams on a curve to achieve a particular grade distribution would not 
be congruent with the collaborative learning in PSS and may undermine students’ learning in pairs 
(Archer-Kath, 2010; Smith, 1996). 

Here we describe the approaches we use when we teach BMED 2210 using the PSS approach. 
Each week the classroom activities focus on a new concept having to do with using mass and energy 
balances to solve problems. In addition to the in-class PSS problems and practice quizzes, students 
are expected to carry out the following learning activities to help them master the material: 1) weekly 
assigned readings from the required textbook or from other sources such as a supplemental text¬ 
book or course notes; 2) weekly homework assignments that reinforce and go into greater depth 
the concepts and problem-solving approaches that were focused on in PSS that week; 3) weekly 
quizzes that focus on the material that was covered in the previous week’s homework assignment; 
4) two mid-term exams; and a 5) cumulative final exam. In general, we do not quiz the students on 
the assigned reading. Instead, we expect students to be motivated to complete the assigned reading 
because they do not want to be left behind by their teammate, and because they know their work is 
publicly visible and will be evaluated multiple times per class by the instructor or the in-class mentor. 


16 


FALL 2016 





ADVANCES IN ENGINEERING EDUCATION 

The Problem Solving Studio: An Apprenticeship Environment 
for Aspiring Engineers 




Figure 3. Key features of the PSS learning environment. 


Figure 3 summarizes the essential elements of the Problem Solving Studio. The overall goal of 
PSS is to have the students in sustained engagement with the course content. This is achieved by the 
three main features: a public, shared problem-solving space, just in time discussions, and situated 
feedback. These features are supported by a variety of structural elements, including reconfigurable 
tables and chairs, blotter pads, complex problems, mentors, tablemates, and partners. The arrows 
denote the primary linkages among items. In addition, we have posted a three-minute video that 
depicts PSS: http://resourcecenters2015.videohall.com/posters/502 . 


METHODS AND RESULTS 


Quantitative analysis of the impact of PSS on students’ conceptual understanding 

We developed PSS to address the need for new learning environments that teach students en¬ 
gineering problem-solving approaches that do not rely on the rote use of memorized algorithms 
and that facilitates deep conceptual understanding. Since the focus of PSS is on the problem solv¬ 
ing process , concerns may arise regarding how well students learn the concepts of the course. To 
investigate this concern, we studied a convenience sample of students who enrolled in BMED 2210 
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Variable 

Number (%) 

Gender 


Males 

64 (47%) 

Females 

71 (53%) 

Ethnicity 


White / Caucasian 

52 (39%) 

East Asian 

28 (21%) 

South Asian 

28(21%) 

Hispanic 

13 (9.6%) 

Black / African-American 

5 (3.7%) 


Table 3. Demographics of study participants. 


during the following semesters: Fall 2012, Spring 2013, Fall 2013, and Spring 2014. Each course was 
taught by one of the authors (J.L.) using the PSS approach. The participants’ demographics are sum¬ 
marized in Table 3. 

We used Shallcross’ Material and Energy Balance Concept Inventory (Cl) to assess students’ 
conceptual understanding of the material early in the semester and again late in the semester 
(Shallcross 2010). Shallcross’ Cl is a 22 multiple choice question instrument that assesses a 
student’s understanding of basic material and energy balance principles. The instrument is 
designed to identify the misconceptions that students have when they start the course, some 
of which are deeply engrained and difficult to correct. We implemented these early and late Cl 
tests in each of the 4 semesters studied. The early Cl scores across the semesters were similar, 
as evidenced by the failure of ANOVA (p = 0.712) and Kruskal-Wallis (p = 0.756) tests to reject 
the hypothesis they had equivalent means. To test the hypothesis that the late Cl scores were 
higher than the early Cl scores, we conducted a paired t-test. The late Cl scores were 3.98 
points higher (out of 22 total points; p < 0.0001) with a 95% confidence interval of [3.45, 4.50], 
See Figure 4 to see a plot of each student’s paired Cl scores. The points are jittered to make 
multiple observations of the same values visible. These results indicate that the PSS students’ 
understanding of mass and energy balances concepts improved significantly over the course 
of a single semester. 

To account for the fact that it is harder for students whose Cl scores were high at the beginning 
of the semester to significantly improve their late Cl score, we also calculated the normalized gain 
of the students’ scores as follows (Flake, 1998): 

(Late Cl Score - Early Cl Score) 

Normalized Gain of Cl Total Score = -* 100 

(22 - Early Cl Score) 
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Early Cl Scores 


Figure 4. Late Cl Scores vs Early Cl Scores. 


The distribution of these normalized gain scores are plotted in Figure 5, which shows that almost 
all of the students improved their Cl scores. A normalized gain score of 14%, 29% and 47% was in 
the 25 th , 50 th , and 75 th percentile, respectively. In addition, we tested the hypothesis that the mean 
Normalized Gain Total Score was greater than zero using a one-sided t-test, which returned a p-value 
of less than 0.0001 with a sample average of 27.68%. 
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Early Cl Scores 


Figure 6. Normalized Gain Scores vs Early Cl Scores. 


We wondered if students who started with a better conceptual understanding were able to 
achieve more of their potential score increase than students who started with a lower score. To 
test this hypothesis, we plotted the Normalized Gain Scores against the Early Cl Scores (Fig¬ 
ure 6). We found no relationship between the Early Cl Score and the Normalized Gain Scores. 
A linear regression on these variables failed to reject the slope coefficient was zero (p = 0.18). 
These findings suggest that no matter what level of conceptual understanding a student has 
when they start the course, they can achieve significant gains in conceptual understanding over 
the semester. 

Finally, we analyzed each semester separately. Figure 7 plots the individual Late Cl Scores 
versus Early Cl Scores for each semester the study was conducted. We see that in each semester 
the majority of students’ late Cl scores were higher than their early Cl scores (as indicated by the 
points that lie above the diagonal line that bisects the plot). These differences were statistically 
significant as indicated by paired t-tests whose p-values were in the range of 0.000005. These 
change scores were quite large for many students, which is indicated by the vertical distance 
from the diagonal line. 

Taken together, these analyses show that PSS significantly improved our students’ conceptual 
understanding of material and energy balances by providing them with a team-based apprentice¬ 
ship studio environment that focused their attention on learning how to solve challenging analytical 
engineering problems. 
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Fall 2012 < n = 43) Spring 2013 ( n = 41 ) 




Fall 2013 ( n = 36 ) Spring 2014 ( n = 44 ) 




Figure 7. Late Cl Scores versus Early Cl Scores for Four Semesters. 


SUMMARY AND CONCLUSIONS 


This paper describes the development of a learning environment we call the problem-solving studio 
(PSS). The PSS approach refers to a set of participant structures that govern how instructors spend 
classroom time with their students. The affordances of PSS make it possible for instructors to carry out 
dynamic scaffolding, which means they are able to modify the support they provide in real-time, as well 
as modify the difficulty level of the problems, to help ensure each student is challenged at a level that is 
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beyond what they could accomplish on their own, but at the upper end of what they can accomplish in 
the PSS setting. Since concerns are sometimes raised that problem-solving does not facilitate growth 
in conceptual understanding, we studied the impact PSS had on students’ conceptual understand¬ 
ing of material and energy balances when it was used to teach an entry-level biomedical engineering 
course on this topic (i.e., BMED 2210 at Georgia Tech). We used Shallcross’ concept inventory of mass 
and energy balances to quantify the gains in conceptual understanding that PSS students’ achieved 
(Shallcross 2010). The average score increase was 4 points out of 22, with a high level of statistical sig¬ 
nificance. We found the average normalized gain score was 28% and independent of the early Cl score. 
The main implication of our findings is that learning environments that focus on developing students’ 
problem solving skills can significantly enhance students’ conceptual understanding of the material. 
We anticipate that instructors who decide to flip their courses will find the PSS approach useful as a 
way to structure the in-class time they have freed up by putting their lectures online. 
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APPENDIX 


Problem Statement 

Textbook problem statement: 

Consider a warm dry day with an air temperature of 28°C and a relative 
humidity of 60% (i.e., has a moisture content of 0.014 g water vapor per g dry 
air). A person sitting on a park bench enjoying the beautiful spring day inhales 
at an average rate of 7 g dry air/min and exhales air saturated with water at 
body temperature (37°C) and 1 atm. How much heat is this person losing by 
breathing? Report your answer in [kcal/hr]. A diagram of this system is depicted 
below and the heat capacity of dry air is 1.05 J/(g °C) 

PSS version of the problem: 

How much energy do we lose per day 
by breathing? Report your answer in 
food calories per day. 

Problem Structuredness 

The PSS problem is 
less well-structured 

because it leaves it to 
the problem solver to 
infer... 

• the temperature and pressure of the exhaled air, which should be possible since most 
students at this level know that the normal core body temperature of humans is 37°C and 
that the barometric pressure is usually close to 1 atmosphere 

• that the air inside the lungs is in contact with moist tissues long enough to become 
saturated with water vapor 

The PSS problem is 
less well-structured 

because it has multiple 
viable solutions 
because... 

• it does not prescribe for the problem solver how to model the inhaled or exhaled gases. 
Therefore, the problem solver could decide to model the incoming air under that day’s 
weather conditions or under any other set of conditions. 

• the problem solver must decide how to model the composition of the exhaled air. Choices 
include solving the problem using realistic compositions of exhaled air that would include 
less oxygen and more carbon dioxide, or using the simplest possible model that treats the 
inhaled and exhaled gases as composed of water vapor and one non-condensable gas such 
as nitrogen. 

Problem Complexity 

The PSS problem 
is more complex 

because... 

• It requires the problem solver to have a greater breadth and depth of knowledge regarding 
human physiology and the chemistry of two-phase systems. 

• For example, the PSS version requires the problem solver to: 

• know, or be able to estimate, the rate at which humans breathe and the average volume 
of each breath 

• recognize that the exhaled gases will be saturated with water vapor, and then be able to 
calculate how much of the exhaled gas is water vapor and how much of it is other gases 


Table 4. An illustration of how PSS problems are less structured and more complex. 


FALL 2016 


27 
















